Gain Chart
더 많은 작업
A Gain Chart, or Cumulative Gain Chart, is a graphical tool used to evaluate the effectiveness of a predictive model by showing the cumulative percentage of positive outcomes identified as more of the dataset is included. It helps assess how well the model ranks positive cases, particularly in applications where targeting high-value instances is essential.
A Gain Chart plots the cumulative percentage of positive outcomes (y-axis) against the cumulative percentage of the population (x-axis), ordered by model predictions. The chart compares the model’s performance to a random baseline, illustrating how much gain is achieved by using the model instead of random selection.
- Steeper Curve: Indicates that the model is effective at identifying positive cases early in the ranked population.
- Random Line Baseline: Represents a scenario where instances are selected without a model, resulting in a diagonal line.
A Gain Chart provides insights into model performance and helps make targeting decisions:
- A steep initial gain shows that the model captures a high proportion of positive outcomes within the first few segments of the dataset.
- The closer the model's gain curve is to the top-left corner, the better the model is at ranking positive instances.
- When the gain curve flattens, it suggests diminishing returns, as fewer positive cases are found in the additional population.
Gain Charts are widely used in areas where prioritizing high-value cases can improve resource efficiency:
- Direct Marketing: Evaluates the model’s ability to identify likely responders within a smaller customer segment, optimizing campaign resources.
- Customer Retention: Determines the highest-risk customers, enabling targeted retention efforts with limited resources.
- Fraud Detection: Assesses which portion of transactions should be flagged for further review, maximizing fraud detection within a small segment.
While both Gain Charts and Lift Curves evaluate model effectiveness, they differ slightly:
- Gain Chart: Focuses on the cumulative percentage of positives captured at different selection levels, illustrating overall performance.
- Lift Curve: Measures the model’s improvement over random selection within each selected segment, providing insights into performance relative to baseline.
Gain Charts are useful tools for model assessment and decision-making:
- Resource Allocation Insight: Helps determine the optimal population size to target for the highest gains.
- Performance Comparison: Useful for comparing different models to see which captures more positive outcomes within the same segment.
While informative, Gain Charts have certain limitations:
- Sensitive to Class Imbalance: Gain can appear exaggerated in highly imbalanced datasets, making it necessary to use additional metrics for context.
- Generalization Challenges: Gain is specific to the dataset used, and results may not generalize to other datasets without similar class distributions.
Gain Charts are often analyzed alongside other tools to provide a comprehensive view of model performance:
- Lift Curve: Complements the Gain Chart by focusing on model improvement over random selection.
- Cumulative Response Curve: Shows the cumulative proportion of positive cases, similar to a Gain Chart but with different interpretations.
- ROC Curve: Useful for evaluating the trade-off between sensitivity and specificity across thresholds.
이익도표(Gain Chart)는 분류 모형의 성과를 평가하는 척도 가운데 하나로, 관측치를 임의의 등급으로 나누어 등급마다 반응검출률, 반응률, 향상도 같은 정보를 산출해 예측이 얼마나 잘 이루어졌는지를 보여 준다. 한국어 교재에서는 Gain Chart, Lift Chart 를 모두 이익도표로 옮기며, 이익도표의 향상도를 그래프로 그린 것을 향상도 곡선(Lift Curve)이라 한다.
- 검증용 데이터의 관측치를 모형이 준 예측 확률(점수)이 높은 순으로 정렬한다.
- 정렬한 데이터를 같은 크기의 등급으로 나눈다. 보통 10개 등급(상위 10%씩, decile)을 쓴다.
- 등급마다 아래 값을 계산한다.
| 지표 | 계산 | 뜻 |
|---|---|---|
| 반응검출률(% Captured Response) | 해당 등급의 실제 반응 수 ÷ 전체 실제 반응 수 | 전체 반응자 중 그 등급이 잡아낸 비율 |
| 반응률(% Response) | 해당 등급의 실제 반응 수 ÷ 해당 등급의 관측치 수 | 그 등급 안에서 반응자가 차지하는 비율 |
| 향상도(Lift) | 해당 등급의 반응률 ÷ 기본 향상도(전체 데이터의 반응률) | 모형 없이 무작위로 뽑을 때보다 몇 배 잘 맞히는가 |
- 좋은 모형이면 상위 등급의 향상도가 1 보다 크게 높고, 등급이 내려갈수록 빠르게 줄어든다.
- 등급별 향상도가 들쭉날쭉하면 모형의 순위 매김이 잘못된 것이다.
- 향상도가 1 이면 무작위 선택과 다를 바 없다.